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ACADEMIC ENHANCEMENT SITE 

J udith A. DeJ ong, Ph.D., and Stanley R. Holder, M.S. 


Abstract: This off- reservation boarding school serves over 600 students in grades 4-12; 
approximately 85% of the students reside in campus dormitories. After having documented 
significant improvement on a number of outcomes during a previous High Risk Youth Prevention 
demonstration grant, the site submitted a Therapeutic Residential Mode! proposal, reguesting 
funding to continue successful elements developed under the demonstration grant and to expand 
mental health services. The site received Therapeutic Residential Mode! funding for school year 
2001-2002. Once funds were received, the site chose to shift Therapeutic Residential Mode! 
funds to an intensive academic enhancement effort While not in compliance with the 
Therapeutic Residential Mode! initiative and therefore not funded in subseguent years, this site 
created the opportunity to enhance the research design by providing a naturally occurring 
placebo condition at a site with extensive cross-sectional data baselines that addressed issues 
related to current federal educational policies. 
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Academic Enhancement Site 

Located less than a mile from a community of approximately 7,000 residents and 
surrounded by wooded areas, this off- reservation boarding school serves approximately 600 
students in grades 4-12. Approximately 85% of students were housed in campus dormitories. 
Prior to Therapeutic Residential Model (TRM) funding, this school had documented significant 
improvement over the course of a Center for Substance Abuse Prevention 
(CSAP/SAMHSA/PHS/DHHS)^ demonstration grant that began in 1995 and ended in 1999. A 
precursor to the TRM approach, this demonstration grant brought the concept of a 
comprehensive, data-driven prevention approach to campus. The school's mode of operation 
shifted during that half-decade from a focus on crisis management to a proactive approach. 
Spearheaded by a project director oriented toward a therapeutic approach and supported by 
reform-minded administrators, the site experimented with a variety of interventions. 

This site pioneered the use of paraprofessional counseling technicians as parenting 
surrogates who provided one-on-one attention, care, and advocacy for students. Several 
specialized residential settings for short-term placement were piloted. These included a 
transition dormitory for students whose lack of social skills resulted in behavioral problems, an 
opportunities dormitory for students in need of academic assistance, and a therapeutic dormitory 
for students with significant mental health needs. During the period that these CSAP prevention 
programs were in place, there was a 50% drop in tobacco use, a decrease from 87% to 50% 
among 7'^'^ and 8‘^ graders in marijuana use, an increase in academic proficiency in math from 
44% in 1997 to 83% in 2000, and an increase in language proficiency from 41% in 1997 to 86% 
in 2000. 

Recognizing that serious problems remained, this site presented a comprehensive TRM 
proposal that was accepted and funded for the school year (SY) 2001-2002. This proposal 
sought to continue and enhance elements developed under CSAP funding, while significantly 
enhancing the mental health resources and other comprehensive services for students. In the 
gap between the proposal submission and receipt of funding, however, the No Child Left Behind 
initiative came into being. When TRM funds were received, the school administration chose to 
shift the majority of that funding to academic enhancement activities. Evaluation efforts 
documented this effort and the outcomes associated with it, thus providing a naturally occurring 
alternate treatment group for comparison purposes. After the first year of funding, the Office of 
Indian Education Programs decided to eliminate the site from the TRM program because its 
proposed therapeutic approach had not been implemented. However, data from this site were 
valuable for comparison purposes. Among other advantages, this site had 5 years of data 
documenting its evolution as a precursor to the TRM. When the data were analyzed for the year 
of TRM funding, the site showed little change in the outcome indicators. Results from this site 
underscore the importance of administration commitment, the necessity for oversight, and the 
need for a multidimensional TRM approach. 

Student Characteristics 

Life Stressors. Table 1 shows responses of Academic Enhancement site (AE) students on 
the Prevention Planning Survey section of the American Drug and Alcohol Survey (ADAS), 
administered anonymously in fall 2001. It is clear that many incoming students had experienced 
school failure and been exposed to violence as either perpetrators or victims. 
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Table 1 

History of Incoming Students in Faii 2001 - 
Anonymous ADAS Seif Reports 


Item 

Grade 

(N=105)* 

gth iQth Qratje 

(N=202)* 

nth 12th 

(N=154)* 

History of Antisocial Activity 

- Have been arrested 

25% 

50% 

41% 

- Have robbed someone 

17% 

22% 

25% 

- Have beaten up somebody 
-Have hurt someone using club/chain/ 

74% 

74% 

64% 

knIfe/gun 

16% 

25% 

19% 

School Failure 

- Have flunked a grade 

47% 

60% 

51% 

- Have been expelled from school 

22% 

28% 

31% 

Victimization 

- Have been beaten up by peer 

- Have been beaten up by someone not 

22% 

28% 

19% 

of same age 

39% 

40% 

38% 

- Been hurt with a club/knife/gun 

21% 

27% 

26% 

- Been robbed 

15% 

15% 

13% 


*Not all students responded to all Items. Percentages are based on number of students 
responding yes or no. 


Suicidal Tendencies . In 1999, the Johnson Student View was administered anonymously 
to grades 7-12 using a paper-and-pencil format. As can be seen in Figure 1, at each grade level 
girls were more likely than boys to report having attempted suicide. Overall, 29% of students 
reported having attempted suicide. 


Figure 1 

Percentage of A E Students Reporting Past Suicide Attempts by Grade and Gender 



American Indian and Aiaska Native Mentai Heaith Research: The Journai of the Nationai Center 
American Indian and Aiaska Native Programs, University of Coiorado at Denver and Heaith Sciences Center ( www.uchsc.edu/ai) 


126 


VOLUME 13, NUMBER 2 


Gang Involvement . Students were asked in fall 2001 surveys about their gang 
involvement. Figure 2 shows the percentage of students choosing each option to describe their 
level of involvement. 



Alienation . Using the short form of the J essor Alienation Scale (J essor, Donovan, & Costa 
Frances, 1992), baseline measures of alienation were taken in fall 2001 at the beginning of the 
year of funding (Table 2). 


Table 2 

Percentage of Students Agreeing with Jessor Items 
in Fall 2001 


Jessor Item 

Grade 

9*^10''' Grade 

nth i2<h Grade 

Hardly anything I’m doing in my life means 
very much to me. 

41% 

33% 

26% 

1 sometimes feel unsure about who 1 really 
am. 

31% 

28% 

27% 

It’s not up to me to help out when people 1 
know are having problems. 

40% 

23% 

27% 

It’s hard to know how to act most of the 
time since you can’t tell what other people 
expect. 

60% 

51% 

48% 


Substance Abuse . Data collected on substance use by the CSAP project showed that a 
long and gritty war with substance use had taken place on campus in the years preceding TRM 
funding. Figures 3-5 show drug use in the previous 30 days at AE for spring and fall data points. 
Marijuana was the drug of choice at this site. Baseline data collected in spring 1996 showed that 
87% of the and graders reported using marijuana in the past 30 days. In the year 
following implementation (SY 1996-1997), fall and spring data showed an increase during the 
course of the school year m the number of students using marijuana. However, the tide turned 
in SY 1997-1998 when marijuana use decreased between fall and spring. By spring 1999, at the 
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end of the CSAP data collection, cross-sectional data showed that the percentage of and 
graders using marijuana in the past thirty days had fallen to 50%; for and 10*^^ graders it had 
fallen from 74% to 52% , and for 11*'^ and 12*'' graders it had fallen from 76% to 59% . 



The pattern of alcohol use shown in Figure 4 is similar to that of marijuana use. The 
number of users increased between fall and spring for SY 1996-1997, indicating that students 
were being inducted into both marijuana and alcohol use during the year. However, the 
numbers for all age groups decreased during subsequent school years. 


Figure 4 

Percentage of AE Students Reporting Alcohol Use in the Past 30 Days 
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Plagued by an employee union that insisted on the right of staff members to smoke 
inside school buildings, the CSAP prevention program struggled to reduce cigarette use by 
students (Figure 5). In SY 1997-1998, a new reform-minded superintendent forced staff to leave 
the campus boundaries in order to smoke.. Figure 5 shows a decrease in tobacco among the 
students after enforcement of this policy. 


Figure 5 

Percentage of AE Students reporting Daily Use of Cigarettes 



BarOn Emotional Quotient Inventory . The BarOn Emotional Quotient Inventory focuses 
on a number of areas necessary for successful functioning in the world, including intrapersonal 
skills (having to do with inner knowledge and inner balance), interpersonal skills, ability to 
manage stress, and adaptability (reflecting confidence in ability to deal with situations that arise 
around oneself). Table 3 shows the percentage of incoming students aged 10-19 scoring in the 
range of low to markedly low on BarOn scales in fall 2001. While generally within the normal 
curve on the Intrapersonal and Stress Management scales, the percentage of students scoring 
low on the Interpersonal and Adaptability scales indicated that many students lack interpersonal 
skills and confidence in their ability to handle challenges. 


Table 3 

Percentage of AE Students Scoring Low to Markedly Low on BarOn Scales 


SCALE 

Males 

(n=207) 

Females 

(n=164) 

Intrapersonal 

28.2% 

31.7% 

Interpersonal 

68.1% 

47.0% 

Stress Management 

14.5% 

28.7% 

Adaptability 

66.7% 

52.4% 
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Site Resources 

Facilities and Resource Management 

The condition of the facilities ranges from adequate to poor. The majority of buildings 
on the campus are over sixty years old, well past the date when they should have been replaced. 
A number of old and beautiful trees shade a campus which is relatively free of trash. There are 
sidewalks and asphalt roads, but they are in poor condition. Negotiating the campus after a 
rainstorm is difficult, as mudslides from the "lawn" cover sidewalks with thick, treacherous mud. 
Potholes on the road circling campus are serious enough to create hazards for walking or driving. 

The school buildings are in generally good condition. There are two main buildings, a 
high school and an elementary building, supplemented by a cluster of modular units. The library 
has a variety of fiction to encourage reading. Bathrooms have inadequate ventilation and need 
repairs. Students and staff report that there is frequently only cold water. Doors have been 
removed from stalls in bathrooms to deter substance use. 

The conditions of the dormitories vary, in the two-year interval between TRM proposal 
submission and receipt of funding, enrollment increased 18%. Student intake assessments done 
by the counseling technicians allowed the school to obtain intensive Residential Guidance (IRG) 
funds for over 99% of students; therefore, school revenue per student and money for services to 
these students further expanded the available budget. Between SY 1999-2000 and SY 2001- 
2002 (the year of TRM funding), the number of staff increased from 140 to 219. 

The increase in students resulted in overcrowding in the dormitories. Old, cottage-style 
buildings housed the younger male students and the majority of female students. There were 
two "newer" dormitories; one housed athletes, and the other housed high school boys. There 
was a disparity in accommodations. Residents of the new dormitories had three students per 
room, while the young women and children in the older dormitories were stacked in bunk beds, 
with up to 18 beds in a large room and 4 to a small room. The bathrooms in the old dormitories 
had cracked tile, mold, and mildew due to lack of ventilation fans. Shower facilities were so 
limited that young women's wake-up calls for shower rotation started at 4 a.m. Lack of 
ventilation was exacerbated on the ground floor, where windows were locked in an attempt to 
control behavioral problems. The appearance of the dormitories was shabby, with windows that 
had pieces of cloth or blankets hung over them. Some attempt had been made on furnishings. 
Attractive new beds were seen in all the dorms, covered by fuzzy blankets with striking American 
I ndian designs. 

At the time of funding, recreational amenities were limited. There was an aging, dusty 
gym that was being designated an historical monument; a small fitness center; and a number of 
outside fields and athletic courts. One new steel building housed the football program. 
Construction had begun on a large, modern gymnasium, the size of which dwarfed the other 
buildings on campus. 

The staff survey in spring 2002 solicited opinions on campus conditions and resources, 
asking staff members to use a 5-point scale ranging from 0 [not a problem here) to 4 [a major 
problem). Figure 6 shows the staff members' ratings of resources. 
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Figure 6 

AE Staff Ratings of Resources (Scale: 0 = not a problem here to 4 = a major problem) 



Activities Limits Service 


"Crowded living conditions in the dorm" generated the most concern among the staff, 
with 75% identifying it as a moderate to serious issue. 

Staff morale was relatively high, with only about one-fifth of the staff rating either "Staff 
dissention" or "Low staff morale" as a major problem. Many staff had a positive "can do" attitude 
and were willing to push forward if they received leadership. Early in the school year, the 
Student Services Department director, a mental health professional who had spearheaded the 
reform effort begun under CSAP funding, left the program. After a period without leadership, the 
position was filled by the recreation director who had no background in mental health. 
Unfortunately, the leaderless period coincided with the year of TRM funding, and the staff had 
little oversight and lacked clear direction. 

The TRM program brought in $1.5 million during the year of funding. Because the site 
combined program funding rather than separately budgeting and tracking TRM funds, the 
evaluation compared resources and staffing patterns in the year of funding with those in the 
preceding years to ascertain changes. 

Family and Community I nvolvement 

Because almost 85% board at the school, many students miss the support of their 
families and communities. Figure 7 shows the results of a staff survey about issues impacting 
students. Most of the staff felt that family issues were the principal source of student problems. 
Unfortunately, given the distance most students are from their families, addressing family issues 
is difficult. Only in the last month of TRM funding did one of the family counseling sessions 
described in the proposal take place. 
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Figure 7 

AE Staff Ratings of Support from the Home 
(Scale: 0=Not a problem hereto 4=M ajor Problem) 
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Staff opinions were evenly spread out on the influence of outside politics. Only one- 
quarter of staff had serious to major concerns regarding "School board policies" (Figure 8). 


Figure 8 

AE Staff Ratings of External Support (Scale: 0=Not a problem hereto 4=M ajor problem) 
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Program and Service Components 


Cultural Programming 

This site places a strong emphasis upon culture. The school recognizes and has pride in 
its history and culture. There is a long-standing American indian Heritage Club. Hand games, 
pow wows, traditional dance and drum groups, traditional crafts, and an American indian Color 
Guard are some of the cultural activities taking place on the campus. Many of the cultural 
activities, which had previously been sporadic and sometimes funded out of the pockets of 
dedicated staff members, were built up by CSAP funding during the previous demonstration 
project. The TRM funding was to provide additional staff and support additional cultural 
activities. 

Students had a relatively high degree of cultural pride, as measured on a cultural pride 
scale. Figures 9 and 10 show cultural pride levels at three time points: spring 2001 (the baseline 
prior to TRM funding), fall 2001 (for incoming students), and spring 2002 (the end of TRM 
funding). 


Figure 9 

Cultural Pride Means for AE Elementary Students 
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Figure B 

Cultural Pride M eans for AE 7-]2th Graders 



Membership Family Peer Sharing Participation Pride Knowledge Legends 
Membership 


The highest levels of pride were on Membership ("I like being a member of my tribe," "I 
like that my family is part of my tribe," and "I am proud to be a member of my tribe"). The 
lowest levels of cultural pride were on Peer Sharing ("I talk to my friends about things having to 
do with my tribe's culture [religion, customs, values, food, language, arts, pow wow and other 
celebrations") and participation ("I participate in tribal and other Indian celebrations"). 
Knowledge ("I know about my tribe's culture and history" and "I like telling and listening to tribal 
legends and stories about my ancestors") was between these two clusters. 

Socialization/ Life Skills Components. 

Elements involved in enhancement of social development and life skills in any school 
environment are made up of proactive (preventive) elements and reactive (disciplinary) policies. 

Proactive Components . 

Counseling Technicians . During the previous prevention funding, the site recognized that 
students were not receiving basic emotional support from dormitory staff, and provided a number 
of prevention staff who worked closely with students. Realizing that students needed role 
models such as these who took on the roles of parents, social workers, and advocates, the school 
created a cadre of paraprofessional Counseling Technicians (CTs) to fill these roles as CSAP 
funding tapered off. The CTs provided assessment of all incoming students, yielding information 
that was used to generate a description of individual needs and strategies for meeting these 
needs. Their role had evolved over the years, and included coordinating services, accessing 
services, acting as a liaison between parents and teachers, and providing life skills training. At 
the beginning of the year of TRM funding, CTs were based in a group office, supervised by an 
experienced lead CT, and had developed a high level of cohesion. Less experienced members of 
the group extensively consulted with their more experienced colleagues when they encountered 
problems with students. However, reorganization in January 2001 (the year of TRM funding) had 
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a major effect on the group, as members lost their office, were reassigned to dormitory duties, 
and had their caseloads shuffled. 

The TRM proposal requested funding to bolster cultural and alternative activities, and to 
fund an art therapist, a music therapist, 17 additional home living assistants, a cultural tech, and 
an additional CT. Funds to support the professional development of staff were also requested. 
Under TRM funding, the number of dormitory and recreational staff increased from 40 to 66, 
improving the student-staff ratio from 12:1 to 9:1 A substance abuse specialist was added to 
provide prevention services. However, the music therapist, cultural tech, and counseling tech 
were not added, and the tenure of the art therapist was brief. As the number of students had 
increased since the year the proposal was submitted, CTs' caseloads were increased instead of 
lightened. 

Reactive Components 

The number of security personnel was increased from two to four under TRM funding. 

Discipline . In surveys in 2001 and in 2002, staff indicated their concern about a 
perceived lack of student discipline and inconsistent treatment of students. In October 2001, 
37% of staff rated student discipline as a major problem; in April 2002, 40% rated it as major. 
In 2001, 39% of staff saw a need for consistency. By April 2002, 45% of the staff rated 
inconsistency as major problem. 

The school had a well-established system of discipline. Staff had a behavioral incident 
form that listed two categories of offenses. Category A offenses, the more serious ones, included 
weapons, fighting, possession of substances or paraphernalia, gang involvement, endangerment 
of self or others, contract violation, felony, AWOL, and sexual misconduct. The school had a no- 
tolerance policy for these Category A offenses. Students cited for Category A offenses went 
before a guidance committee. If found guilty, they were given the choice of being suspended, 
withdrawing, or going to the transition dorm. This dorm provided a strictly regimented 
environment that in past school years included removal from the classroom and working with a 
special group of Student Services Department teachers. During TRM funding, however, the 
transition students were mainstreamed back into the classroom. 

A tracking system for behavioral incidents had been put in place under CSAP funding. 
The elementary school counselor entered behavioral incidents that came before the committee, 
and disciplinary actions based on the committee's decisions, into the school computer system. 
The loss of the director of student services, a strict disciplinarian, appeared to affect this system. 
The evaluator received a number of reports indicating that staff were being discouraged from 
making reports on students, and that some reports that staff tried to file were being shelved in 
an effort to circumvent the system. Almost one-half of the students appeared to believe that 
there was discrimination in enforcement of rules. When asked whether it was either "very true" 
or "mostly true" that school rules were fairly enforced, only 52% of 7'^^ and B*^*^ graders, 58% of 
9‘^ and 10'^*^ graders, and 55% of 11‘^ and 12'^'^ graders agreed. As many informants suggested 
that athletes were one of the groups being favored, data from the spring 2002 ADAS survey were 
examined, dividing respondents into those who indicated that they were engaged in school 
athletics and those who did not. As Table 4 shows, on items having to do with discipline, self- 
identified athletes were consistently less likely to believe they would get in "a lot" of trouble for 
violating rules. 
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Table 4 

Enforcement of Standards (ADAS Items) 





Non- 

Athletes 

Athletes 

Percentage of students believing students get in 
for cutting school. 

“a lot" 

of trouble 

45% 

28% 

Percentage of students believing students get in 
for bullying another student. 

“a lot" 

of trouble 

51% 

34% 

Percentage of students believing students get in 
for hitting another student. 

“a lot" 

of trouble 

52% 

47% 

Percentage of students believing students get in 
for smoking. 

“a lot" 

of trouble 

77% 

60% 

Percentage of students believing students get in 
for drinking alcohol. 

“a lot" 

of trouble 

73% 

61% 

Percentage of students believing students get in 
for using other drugs. 

“a lot" 

of trouble 

73% 

63% 


Staff surveys rated "Drug and alcohol problems of students" on the scale of 0 {not a 
problem here) to 4 {a major problem) in spring 2001 and spring 2002. In spring 2001, 29.5% 
rated this item as a major problem; in spring 2002, 35.1% rated it as a major problem. 

Student agreement with the Jessor item "It’s hard to know how to act most of the time 
since you can't tell what other people expect" is shown in Figure 11. While the small number of 
students in the youngest age group (20 in spring 2001; 27 in spring 2002) precludes statistical 
conclusions, there appears to be a decrease in agreement with this item, indicating that they may 
have found the environment more predictable than had their predecessors. 


Figure H 

Percentage of AE Students Agreeing: "It's hard to knowhowto act most of the time since you can't tell 

what other people expect." 
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Mental Health Services 

Figure 12 shows that staff recognized the mental health needs of the students. All staff 
respondents indicated that students had some degree of emotional problems. All staff agreed 
that students' low expectations of themselves, and their alcohol or drug use, were significant 
problems. 


Figure ]2 

AE Staff Ratings of Student Mental Health Needs 
(Scale: 0=Not a problem hereto 4=Major Problem) 
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In the TRM proposal, mental health was described as an area with a high level of need, 
requiring a significant number of trained personnel. The proposal described a system of triage 
that would use counseling technicians (CTs) at the base level of triage to assess all students for 
mental health needs. Higher levels of triage would use additional professional staff to deal with 
students identified by the base assessment as needing more help. In addition to these 
professionals, AE proposed to further develop the therapeutic dorm pilot project to provide an 
appropriate setting for students with serious mental health needs requiring the highest, or level 
three triage. 

As described by the proposal generated in SY 1999-2000, AE's original resources in the 
mental health area included an elementary school counselor, a high school counselor, a half-time 
licensed professional counselor, and a director of student services with a Masters degree in 
counseling. Some very minimal additional assistance was provided by contracts with outside 
sources. The proposal specified two additional CTs would be hired to lighten caseloads on base 
triage, an expansion from 2.5 FTE to 10 FTE in professional level positions (i.e. masters level and 
licensed counselors) to serve students whose mental health needs were beyond the expertise of 
the CTs, and an expansion of the capacity of in-house and consultant services for the most 
serious needs. 

However, as the proposal was implemented, it was found that TRM funds had not been 
not used to increase mental health resources. The number of CTs stayed constant at 13 even 
though the number of students increased, driving the student-CT ratio from 37:1 to 44:1. The 
second level of triage had two school counselors and one Masters-level counselor on staff during 
the year of funding. The school continued to rely on outside counseling contracts, rather than 
providing the 1.5 FTE psychologist and psychiatrist as requested in the proposal. There was 
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significant concern among staff that the half-day per month of psychiatric services provided by 
contracts was inadequate to deal with the problems of many students. This shortcoming is 
particularly significant because 29% of students reported having made a suicide attempt in the 
past. While CTs in the dorms continued to provide some paraprofessional services, they spent 
much of the critical first two months of school either driving buses or filling out paperwork 
required to qualify students for IRG funding. 

Therapeutic Dormitory . One of the proposed additions was a therapeutic dorm to 
provide an appropriate setting for students with serious mental health needs, who would qualify 
for inpatient mental health services. This innovation was unique among the three sites in the first 
cohort, all of whom concluded that many of their students had such serious mental health 
problems that it was necessary to send some of them elsewhere. A very promising model for the 
proposed therapeutic dormitory was developed by a counselor at this site. However, the 
administration initially decided not to implement this component. Midway through the school 
year, the administration reversed its decision and proceeded with implementation. At the time of 
the April site visit, the dormitory had been operating for several months and had served a total of 
20 students. Concerns were expressed that: as additional professional staff had not been hired 
personnel assigned to the therapeutic dormitory were volunteers with no special qualifications or 
training, adequate protocols were not in place to guide personnel in their unfamiliar new roles, 
the dormitory was not supervised by a person holding a Ph.D. or an M.D., and the number of 
mental health professionals on staff overall at the site had decreased since the beginning of the 
year. As the energies of the remaining mental health personnel were being focused on 
operations at the therapeutic dorm, the resources to deal with the needs of other students were 
further compromised. 

Academic Program 

When the TRM proposal was submitted, academics appeared to be adequately 
addressed. According to the SY 1999-2000 school report card, the school had produced a drastic 
improvement in students' academic proficiency. Eight-three percent of the students were 
proficient or above in math and 86% were proficient or above in language in SY 1999-2000. 
Therefore, the TRM proposal requested only four additional teachers to expand services in the 
special needs dorms (transition, opportunities, and therapeutic). This resulted in a teacher- 
student ratio of 1:15.3, much lower than the 1:20 ratio that existed at the comparison day 
school. In addition, a local university agreed to supply unpaid education majors to tutor and to 
assist in classrooms. A structure was in place to identify and triage students in need of academic 
help. Students scoring below a C in any course were required to report to the teacher of that 
course from 3:30 - 4:30 p.m. According to the proposal, the opportunities dorm provided an 
intensive learning environment for students failing one or more subjects. Therapeutic 
Residential Model funding augmented this existing system by providing the services of Alpha Plus 
(discussed in more detail later in this section) for staff development. 

However, test data from 2001 and 2002 painted a less optimistic academic picture. 
Figure 13 shows that many AE students were performing well below their grade level in spring 
2001. Furthermore, the gap in academic achievement widened from two years for 4^^ graders to 
five years for 12^*^ graders. 
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Figure B 

Grade Equivalent Performance of AE Students in Baseline Year (4/01) 



Figure 14 shows results for fifth and eighth graders on the spring 2001 state proficiency 

tests. 


Figure M 

Distribution of A E 5th and 8th Graders on State Proficiency Tests 3/01 
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Figures 15 and 16 show Iowa Test of Basic Skills data collected in October 2001 on 
incoming students. The data show that, at the beginning of the academic year, a significant 
number of students needed remedial academic help. 


Figure B 

Percentage of AE Elementary and M iddle School Students Scoring at or above Fully Proficient byNCE 

Standards 




During the year of TRM funding there were a number of changes to the academic 
program. 

The majority of TRM funding used for academics at this site went to an Alpha Plus 
program and the addition of a large cadre of academic classroom aides, called ACAs. The Alpha 
Plus Learning System follows the principles promoted by Lee Jenkins (2003, 2004). Under this 
system, teachers were required to customize their curriculum, focusing on state objectives for 
each grade level. Under the previous CSAP funding, earlier versions of Alpha Plus study skills 
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modules had been tested for effectiveness. Figure 17 shows the relationship between individual 
dosage of Alpha Plus and change in GPA from CSAP reports. In all areas except social studies, 
GPA appeared to decrease, rather than increase, as a function of increasing dosage. 


Figure 17 

Change in GPA by Level of Alpha Plus Dosage During CSAP Funding SY96-97 



A number of reasons were postulated to explain these unexpected findings. The amount 
of time taken up by Alpha Plus may have cut into the time available for students to do their 
homework. The encouragement given to students who may otherwise have dropped out of 
school, may have led to their staying in school and further decreasing the overall GPA. Observers 
noted that a number of staff administering the curriculum were poorly trained, and 
implementation may have been an issue. As a result of these findings, the school decided to 
scale back Alpha Plus to only those students who needed academic help. At the beginning of 
TRM funding in fall 2001, however, the school turned again to an upgraded Alpha Plus program 
to make a concerted effort to reform its academic system and increase scores on proficiency 
tests. At the beginning of the year, under the leadership of an Alpha Plus coordinator, teachers 
laid out objectives to match state proficiency testing standards. Alpha Plus trainers worked 
extensively with teachers throughout the year to orient them to state test criteria and to improve 
students’ study skills. The focus shifted to "teaching to the test," with teachers required to 
develop classroom plans to cover one or more test-related objectives per day. 

In order to allow teachers to focus more intensively on subjects, the school changed to a 
block schedule. This change gave teachers fewer - but longer - class periods. According to 
ACAs and a focus group with students, some of the teachers did not adjust to this change. They 
continued to do lectures and homework assignments at the previous length, leaving students to 
their own devices for the remainder of the time in the block. Teachers expressed frustration that 
they were trying to teach criterion-level information when their students were too far below 
grade level to understand the material. The pace left little time to remediate students' academic 
deficits. Adding to the stress, midway through the year, the Iowa Test of Basic Skills replaced the 
state test as the proficiency standard, and teachers were required to reorganize their schedule of 
objectives to meet the new criteria. In addition, the Morningside program - designed to address 
gaps in basic skills - was introduced late in the school year, but teachers had little enthusiasm or 
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energy left by that time. Teachers reported a highly stressful year, which was rendered tolerable 
only by the presence of the ACAs who acted as classroom aides, assisted with classroom order, 
and eliminated the need for the school to hire substitute teachers. 

The addition of 85 students between SY 1999-2000 and SY 2001-2002 was matched by 
an increase of only 3 certified teachers. The bulk of TRM funding went to hiring 30 additional 
educational aides, bringing the number to 34. The majority of the new aides were classified and 
paid as CTs and referred to as ACAs. Among this group, some individuals were in graduate 
programs, some were social workers, and a few were teachers and nurses. These individuals 
had more education than the typical classroom aide and were being paid considerably more than 
other classroom aides. Discussions with students and the ACAs indicated that most of the ACAs 
were not being optimally used: 

• A focus group of high school students identified only one CT who they had observed 
doing anything other than running the occasional errand. 

• Some ACAs reported that their supervising teacher did not know what to do with them 
and had them doing little other than photocopying and grading papers. 

• Some ACAs with psychology backgrounds had been told it was not their job when they 
attempted to counsel students asking for help. 

• While several ACAs said they were working hard and were needed in their assigned 
classroom, most of the ACAs expressed the opinion that their abilities were not being 
used. 

• Several ACAs reported that the teachers to whom they were assigned took advantage of 
their presence to take time off, using them as substitute teachers. One ACA had acted 
as a substitute teacher for 30 hours during the past four weeks. 

The opportunities dorm that had been providing remedial academic help in previous 
years was eliminated in favor of intensive study sessions. Students failing any subject were 
required to attend study sessions from 4:30 - 8:30 p.m., Monday through Thursday. Site visits 
to these sessions, however, revealed that less than one-quarter of the students who were 
required to attend were actually present. The school also initiated a policy of not awarding failing 
grades in an attempt to build self-esteem. Students were allowed to redo assignments in study 
hall to bring inadequate homework up to a C level. Teachers expressed some concern that 
students were being given the unrealistic message that they would have unlimited chances to 
redo work and, therefore, that they did the minimum. 

Career Guidance . Lack of career guidance had been an ongoing problem at AE. A 
computer program to provide information about post-secondary options to students was ordered 
but was not implemented during the year of funding. 

Tutors . The evaluator found no evidence that the tutors provided through the state 
university programs were being used. 

Physical Health Components 

The physical needs of the students were being met. Food service in the cafeteria 
provided adequate nutrition at three meals a day, seven days a week. A school nurse was hired 
with TRM funding. 
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The school had athletic fields and a gym. The proposal had identified significant gaps in 
recreational activities and requested an additional four recreation staff, but only one was hired. 
Students complained that there was a significant lack of weekend activities. Many found the 
intensive Monday through Thursday study schedule to be a bore, with very few activities 
scheduled on weeknights. Twenty percent of the staff reported that "lack of after school and 
weekend activities" was a major problem. Many students were observed to be aimlessly 
wandering the campus when they were not studying. 

Outcome Data 

Outcome data tracked a number of indicators. Retention rate was considered the major 
indicator of a successful program. In addition, quantitative data were used to evaluate such key 
indicators of developmental success as school bonding, peer and social bonding, adaptability and 
stress management, meaning and identity, and academic achievement. 

Retention Rate . Figures 18 and 19 compare retention in SY 2000-2001 and SY 2001- 
2002. Both analyses are based on the population that enrolled on or before September 30 of the 
school year. As some 12^*^ grade students graduated midyear, the retention analysis looked only 
at grades 4-11. 


Figure 18 

YearlyRetention of Original Cohort byGrade at AE 



Figure 19 diagrams the cumulative percentage of dropouts from this cohort by month. 
No difference can be seen in the retention rates before and after TRM was implemented. The 
sharpest increase in the dropout rate occurred during in January 2002, coinciding with a 
reorganization of four dormitories, the shifting of caseloads, and the diminution of the residential 
CTs' roles. There was no significant difference in retention between the baseline year of SY 
2000-2001 and SY 2001-2002. 
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Figure 39 

Cumulative Dropout of Original Cohort of AE Students by Month 
before (OO-O:^ and during (01-02) TRM Funding Year 



School Bonding 

Students' Attitude toward School . The students' attitude toward school was tapped 
prior to any intervention (1996), after the CSAP funding (2001), and at the end of the TRM 
Project (2002) by asking them to respond to the item "I like school." Figure 20 shows the 
distribution of their responses. 
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Figure 20 

AE Student Responses to Item: "1 like school" 
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Figures 21 and 22 show student responses in spring 2001 prior to TRM funding, and 
spring 2002 at the end of the year of TRM funding, on two ADAS items: "My teachers like me" 
and "My teachers respect me." 




Responses show the majority of students felt they were liked and respected by their 
teachers "some" or "a lot." There was a shift in the seventh-eighth grade group toward feeling 
more respected and liked by their teachers between spring 2001 and 2002. As the teaching 
personnel had not changed between these two time points, the improvement may be due to the 
presence of the additional classroom aides who both assisted and related to the students, as well 
as providing assistance to harried teachers. 
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Peer and Social Bonding 

Alienation. Figure 23 shows cross-sectional percentages of students responding 
affirmatively to a Jessor item regarding feeling "left out." Feelings of being left out trended 
toward a slight decline for the youngest age group from spring 2001 to spring 2002. 


Figure 23 

PercentageofAE Students Agreeing: "i often feei left out of things that other kids are doing." 



Interpersonal . Figure 24 shows the percentages of students scoring low to markedly low 
on the BarOn Interpersonal scale. There was no decrease in the percentage of students needing 
help in this area. While the youngest and oldest students had similar percentages for males and 
females, analysis of spring 2002 scores found that the middle age group had a gender gap, with 
males at 78% and females at 38% . 


Figure 24 

Percentage of AE Students Scoring Lowto Markedly Lowon BarOn Interpersonal Scale 
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Social Responsibility . Figure 25 shows that there was no change in percentage of 
students agreeing with the statement "It's not up to me to help out when people I know are 
having problems." 


Figure 25 

Percentage of A E Students Agreeing: "It's not up to me to help out when other people are having 

problems." 



Behavioral Indicators . The TRM program began during a period when the site had a 
stable behavioral situation. At the beginning of CSAP funding five years previously, students 
reported fights occurring on a daily basis, while school attendance was not enforced and 
incidents were carelessly tracked. Implementation of the CSAP tracking system to monitor and 
appropriately discipline students, and use of CSAP-funded staff to monitor halls and provide 
prevention activities, had helped to turn this situation around. By the end of CSAP funding in 
1999 the number of violent incidents was low, given the population level of 500-600 students, 
and it changed little across the following interval year and TRM year. Sixty violent incidents were 
reported in SY 1999-2000, 62 in SY 2000-2001, and 52 in SY 2001-2002. Alcohol use, previously 
shown in Figure 4, increased from spring 2001 to spring 2002, going from 23% to 31% in and 
graders in cross-sectional analysis, from 17% to 29% in 9*^^ and 10^*^ graders, and from 25% 
to 38% among 11‘^ and 12‘^ graders. 

Prior to the baseline measure in spring 2001, the campus had lost its drug dog and 
experienced a steep upsurge in marijuana use among the younger students. After the dog was 
replaced and security was doubled, marijuana use decreased from 68% to 50% among and 
8‘^ graders, from 54% to 49% among 9^'' and 10‘^ graders, and from 46% to 38% among ll‘^ 
and 12‘^ graders (see Figure 3). 

The enhanced surveillance may also account for a decrease in cigarette use in the middle 
age group of students during SY 2001-2002 (previously shown in Figure 5). 

Adaptability and Stress Management. 

Figure 26 shows the percent of students whose responses on the BarOn in spring 2001 
and spring 2002 revealed a lack of confidence in their ability to deal with situations. While the 
percentage of students scoring low was similar for males and females in the middle age group. 
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there were gender differences for both the youngest (males 31%, females 75%) and oldest 
(males 60%, females 35%) students, indicating an age x gender interaction. Older females 
gained in their confidence in their ability to meet challenges, while older males’ confidence 
decreased. 


Figure 26 

Percentage of AE Students Scoring Low to Markedly Lowon BarOn Adaptability Scale 



Similar percentages of male and female students scored low on stress management. 
Overall, there was no change on this measure from spring 2001 to spring 2002 (Figure 27). The 
percentage of students at this site who scored low on stress management is similar to that of 
their same-age peers in the general population. 


Figure 27 

Percentage of A E Students Scoring Low to Markedly Lowon BarOn Stress Management Scale 
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Meaning and I dentity 

Figures 28 and 29 show the percentage of students agreeing with the Jessor items 
"Hardly anything I'm doing in my life means very much to me" and "I sometimes feel unsure 
about who I really am." There was no change on either measure during the period of the TRM 
project. 


Figure 28 

Percentage of A E Students Agreeing: "Hardly anything I'm doing in my life means very much to me." 



Figure 29 

PercentageofAE Students Agreeing:"! sometimes feel unsure about who I reallyam." 



Figure 30 shows the percentage of students in spring 2001 and spring 2002 scoring low 
to markedly low on the BarOn Intrapersonal scale. As fall baselines were not available for both 
years and as all data were anonymous on these surveys, it was not possible to ascertain why 
there were shifts in spring outcomes. 
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Figure 30 

Percentageof A E Students scoring Low to Markedly Low on BarOn Intrapersonal Scale 



Academic Achievement 

Figure 31 shows proficiency levels after the last year of CSAP funding (SY 1999-2000), 
the interval year of SY 2000-2001, and the year of TRM funding (SY 2001-2002). The number of 
students scoring in the partially proficient range increased significantly in the interval year 
compared with the last year of CSAP funding, although there may have been some questions 
about inclusiveness of the testing in those years. When SY 2000-2001 and SY 2001-2002 were 
compared, there was a slight shift upward in language but a matching shift downward in math. 
Overall, the data support the teachers' perceptions that teaching on-grade objectives left them 
no time to remediate students' deficits in math. 
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Discussion 

This site had a number of advantages. The prior existence of an innovative prevention 
program on campus had motivated both staff and students. The proposal submitted by this site 
contained a number of innovative elements that had been piloted and had shown promise. 
However, under TRM funding most of these elements were not developed and in some cases 
were cut back or eliminated. 

Therapeutic Residential Model funding with resultant program changes did not increase 
mental health services. Because of changes to the administrative team and the loss of the key 
architect of the TRM proposal, program changes were at odds with the needs presented in the 
proposal and with the strategies proposed to deal with those needs. Instead, administrative 
decisions made during the year of TRM funding reflected that belief that an emphasis on athletics 
and academics was the best remedy for dealing with mental and behavioral problems. 
Significant TRM funds went into the classroom and also into a "boot camp" approach to the 
remediation of academic deficits. From the perspective of a research design, this situation was 
fortuitous. The site functionally qualified as a placebo condition. Because of its inclusion in the 
baseline and outcome data-gathering system, and the sizable infusion of supplemental TRM 
funds, it was possible to use this site to examine post-hoc hypotheses. Namely, if sites are given 
supplemental funds to be used at their discretion, that are equal to those per capita given to sites 
implementing approved programs, is there a concomitant improvement in outcomes? The 
findings showed: 

• No significant increase in retention 

• No increase in cultural pride 

• No decrease in alienation related to identity or meaning 

• Mixed results in substance use. Substitution of interdiction for prevention and 
counseling appeared to reduce marijuana and cigarette use, but alcohol use 
increased. 

• No change in percentage of students in need of help with intrapersonal skills, 
stress management, or adaptability. 

• No overall improvement in academic proficiency. 

Findings at this site indicate that increasing revenue to sites without increasing mental 
health resources results in no change in the critical outcome measures of student success. These 
findings underscore the belief that no positive changes can occur in residential boarding schools 
unless the mental health needs of the students are addressed. 
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Footnote 

^ CSAP = Center for Substance Abuse Prevention 
SAMHSA = Substance Abuse and Mental Health Services Administration 
PHS = Public Health Service 
DHHS = Department of Health and Human Services 



